feat: add git submodule support by bjornrobertsson · Pull Request #485 · coder/envbuilder

bjornrobertsson · 2025-12-11T13:28:22Z

Add submodule cloning support via native Go implementation

Summary

This PR resolves/mitigates #74

Changes

Since the existing submodule support in git.go depends on execution of the git binary, this implementation circumvents that limitation by traversing the tree to clone submodules natively in Go.

Behavior

Functionality can be toggled via a parameter (ENVBUILDER_GIT_CLONE_SUBMODULES)
Only runs on initial workspace start
Does not repeat on subsequent starts (unless the repository is deleted)

Add support for initializing and updating git submodules during repository cloning.

Added GitCloneSubmodules option to support recursive submodule initialization.

Resolved issues with git submodule handling for better cloning and URL resolution.

Resolved issues with git submodule handling to ensure robust cloning and URL resolution without relying on git binary calls.

*test.go file addendum

Updated test

Adding tests for Submodules and two missing

bjornrobertsson · 2025-12-11T13:36:41Z

I have read the CLA Document and I hereby sign the CLA
recheck

johnstcn

Thanks for working on this @bjornrobertsson !
I have some comments below, and I think it would also be worthwhile adding an integration test (see: integration/integration_test.go and testutil/gittest/gittest.go).

git/git.go

johnstcn · 2026-01-02T09:13:34Z

git/git_test.go

+		{
+			name:      "absolute",
+			parentURL: "https://example.com/org/main.git",
+			subURL:    "https://github.com/other/repo.git",
+			expect:    "https://github.com/other/repo.git",
+		},
+		{
+			name:      "relativeSibling",
+			parentURL: "https://example.com/org/main.git",
+			subURL:    "../deps/lib.git",
+			expect:    "https://example.com/org/deps/lib.git",
+		},
+		{
+			name:      "relativeChild",
+			parentURL: "https://example.com/org/main.git",
+			subURL:    "./extras/tool.git",
+			expect:    "https://example.com/org/main.git/extras/tool.git",
+		},
+		{
+			name:      "badParent",
+			parentURL: "://bad",
+			subURL:    "./child",
+			expectErr: "parse parent URL",
+		},


Please also test with the URL schemes from TestSetupRepoAuth:

SSH with scheme (ssh://host.tld/repo)

SSH without scheme (git@host.tld:repo/path)

Git scheme (git://git@host.tld:repo/path)

Absolute local path (/path/to/repo)

Relative paths without ./.. (path/to/repo)

Testing SSH and Git revealed negative user experience if ssh:// and git_username were passed. Lead to: #486
Otherwise testing are fine.
The location when extracted leads to the need to work around relative paths, a lot of work and testing led to the current methods as working

git/git.go

README.md

johnstcn · 2026-01-23T10:28:52Z

git/git.go

+}
+
+// initSubmodules recursively initializes and updates all submodules in the repository.
+func initSubmodules(ctx context.Context, logf func(string, ...any), repo *git.Repository, opts CloneRepoOptions) error {


I'm always a bit wary about recursive functions. I'd feel better about this if we had a maximum depth of which to recurse. I'd wager that most repos won't need more than 2 iterations. If ENVBUILDER_GIT_CLONE_SUBMODULES were changed to accept a positive integer instead, this could function as the number of times to recurse.

Changed the acceptance to 0-10, true, false - and added a Depth testing

johnstcn · 2026-01-23T10:30:40Z

testutil/gittest/gittest.go

+		parentSrv = httptest.NewServer(opts.AuthMW(NewServer(fs)))
+	}
+	return parentSrv, submoduleSrv
+}


I'm not 100% sure this needs to be its own function. I think we could probably in-line this into the relevant test for now.

johnstcn · 2026-01-23T10:35:51Z

git/git.go

+	if opts.Submodules {
+		err = initSubmodules(ctx, logf, repo, opts)
+		if err != nil {
+			return true, fmt.Errorf("init submodules: %w", err)
+		}
+	}


I just saw this -- apaprently go-git supports this natively?

https://pkg.go.dev/github.com/go-git/go-git#CloneOptions
https://pkg.go.dev/github.com/go-git/go-git#SubmoduleRescursivity

This seems like the preferable option to me vs bespoke implementation. 👍🏻

That function breaks because of the relative path, and reason for handling this manually (it was confounding and took some time).

Oh, and v6 renames it, fixes the typo but we're not at v6 yet.

I take it you're referencing this commit? We could fork and backport this fix to v5 until v6 comes out. WDYT?

There is nothing concrete that the relative path issues are resolved, more they're not. Checks indicate that there are known bugs with:

HTTPS URLs losing a slash

Multiple relative paths (../../submodule.git) not working

mafredri · 2026-01-23T10:41:23Z

git/git.go

+	if opts.Submodules {
+		err = initSubmodules(ctx, logf, repo, opts)
+		if err != nil {
+			return true, fmt.Errorf("init submodules: %w", err)
+		}
+	}


This seems like the preferable option to me vs bespoke implementation. 👍🏻

mafredri · 2026-01-23T10:46:19Z

docs/env-variables.md

 | `--git-clone-depth` | `ENVBUILDER_GIT_CLONE_DEPTH` |  | The depth to use when cloning the Git repository. |
 | `--git-clone-single-branch` | `ENVBUILDER_GIT_CLONE_SINGLE_BRANCH` |  | Clone only a single branch of the Git repository. |
 | `--git-clone-thinpack` | `ENVBUILDER_GIT_CLONE_THINPACK` | `true` | Git clone with thin pack compatibility enabled, ensuring that even when thin pack compatibility is activated,it will not be turned on for the domain dev.zaure.com. |
+| `--git-clone-submodules` | `ENVBUILDER_GIT_CLONE_SUBMODULES` |  | Recursively clone Git submodules after cloning the repository. |


We may also want to add --git-clone-submodules-depth to limit recursion depth.

WDYT about --git-clone-submodules-depth=0 as "don't clone submodules (default)"?

Works for me 👍🏻

I just add conditional string eval, default, and 0 is 'false' and a number is Depth?
Transitional, it might be nicer if 'true' opens up a sub-menu for Depth in the template, poc:

validation { regex = "^(true|false|[0-9]|10)$" error = "Must be 'true', 'false', or a number from 0-10." } # and # "ENVBUILDER_GIT_CLONE_SUBMODULES" : tostring(local.submodule_depth),

- Added SubmoduleDepth custom type that accepts 'true' (depth 10), 'false' (0), or positive integer - initSubmodules now tracks current depth and stops at max depth - Default is 0 (disabled) - submodules are not cloned unless explicitly enabled

bjornrobertsson added 9 commits December 9, 2025 16:12

Implement submodule initialization in CloneRepo

81b6fd7

Add support for initializing and updating git submodules during repository cloning.

Add GitCloneSubmodules option for cloning

5de553d

Added GitCloneSubmodules option to support recursive submodule initialization.

Fix typo in environment variables documentation entry

adbdccf

Fix submodule handling in git

18384d5

Resolved issues with git submodule handling for better cloning and URL resolution.

Fix submodule handling in README

4d3e529

Resolved issues with git submodule handling to ensure robust cloning and URL resolution without relying on git binary calls.

Update README.md

e183aff

Add ResolveSubmoduleURLForTest function

2d1df11

*test.go file addendum

Add tests for ResolveSubmoduleURL and CloneOptions

5407a5b

Updated test

Add tests for new environment variables in CLI

463fd91

Adding tests for Submodules and two missing

bjornrobertsson linked an issue Dec 11, 2025 that may be closed by this pull request

Git submodule support #74

Open

bjornrobertsson requested a review from johnstcn December 11, 2025 13:29

make gen

04f798b

johnstcn reviewed Jan 2, 2026

View reviewed changes

johnstcn assigned bjornrobertsson Jan 2, 2026

bjornrobertsson added 2 commits January 22, 2026 17:33

Rename ResolveSubmoduleURLForTest to ResolveSubmoduleURL

db5e40d

Add integration test for git submodules

223d3e4

johnstcn mentioned this pull request Jan 22, 2026

fix: improve git URL parsing #486

Merged

bjornrobertsson requested a review from johnstcn January 23, 2026 09:34

johnstcn reviewed Jan 23, 2026

View reviewed changes

mafredri reviewed Jan 23, 2026

View reviewed changes

bjornrobertsson added 3 commits January 23, 2026 11:14

Add tests for submodule depth values

8134964

Remove submodule section from README (belongs in release notes)

442477e

bjornrobertsson requested review from johnstcn and mafredri January 23, 2026 13:13

bjornrobertsson added 4 commits January 23, 2026 15:53

Fix typo: dev.zaure.com -> dev.azure.com

fb31cac

Fix formatting (gofmt alignment)

68ace4b

Fix git_test.go: use SubmoduleDepth int instead of Submodules bool

149ae9a

Update golden file for CLI output test

3f81838

bjornrobertsson added 2 commits February 3, 2026 11:26

Merge branch 'main' into 74-git-submodule-support

ea25253

Fix: assign repo from CloneContext (was discarded after merge)

cb11759

johnstcn approved these changes Feb 6, 2026

View reviewed changes

Merge branch 'main' into 74-git-submodule-support

76138b0

johnstcn changed the title ~~74 git submodule support~~ feat: add git submodule support Feb 6, 2026

Merge branch 'main' into 74-git-submodule-support

79d1fc5

Conversation

bjornrobertsson commented Dec 11, 2025

Add submodule cloning support via native Go implementation

Summary

Changes

Behavior

Uh oh!

bjornrobertsson commented Dec 11, 2025

Uh oh!

johnstcn left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants